After completing this lesson, you’ll be able to:
Assessing the performance of a workspace such as this:
...requires a bit of time and attention, but knowing how a workspace performs helps to make informed decisions on how to best design (or redesign) it.
When testing for performance, ensure that only one instance of FME is running and that all other non-essential programs are closed. You'll also need to turn off feature caching, which can affect the results. Finally, you must turn on log timestamping under Tools > FME Options > Translation > Log Timestamp Information.
Improving the efficiency of a reader requires an estimate of how well it works in the first instance, yet it can take a lot of work to separate the reading speed in a workspace that also transforms data.
The key message that signifies the reading is complete is “Emptying Factory Pipeline.” Here, for example, the reading of the data finished after 26.4 seconds of processing (of course, the actual elapsed time might be longer if FME was waiting for a database or the file system to respond):
2024-03-29 11:48:54| 26.4| 0.0|INFORM|Emptying factory pipeline
However, sometimes, that message can be misleading. FME processes the data at the same time it is reading it, so it won't read an entire data set before processing. To avoid this confusion, disable all the transformers and run only the readers:
With the transformers disabled, the message now appears much sooner:
2024-03-29 11:49:35| 1.2| 0.0|INFORM|Emptying factory pipeline
So, we can tell that reading the data only takes 1.2 seconds.
As with readers, you can only improve a writer's performance if you can first assess how well it is already performing. However, assessing the writing speed has the same complexity as reading: FME starts writing data as soon as it becomes available and doesn’t necessarily wait until processing is done.
So, how can we assess writer performance? Assessing a writer is different from a reader. If you isolate the writers by disabling everything else, there won't be any data to write!
The easiest way is to disable the writer itself!
If the original result was this:
2024-03-29 11:54:14| 26.5| 0.0|INFORM|Translation was SUCCESSFUL with 19 warning(s) (148 feature(s) output) 2024-03-29 11:54:14| 26.5| 0.0|INFORM|FME Session Duration: 26.7 seconds. (CPU: 22.5s user, 4.0s system)
...and with the writer disabled, it is now this:
2024-03-29 11:56:48| 26.6| 0.0|INFORM|Translation was SUCCESSFUL with 19 warning(s) (0 feature(s) output) 2024-03-29 11:56:48| 26.6| 0.0|INFORM|FME Session Duration: 26.6 seconds. (CPU: 22.1s user, 4.4s system)
...then we can easily calculate that the writing process is taking merely 0.1 seconds.
Another method is a two-step process.
First, add a Recorder as the final transformer to store the data in FFS format right as it is about to be written. Run the workspace.
Then, replace the Recorder with a Player transformer to re-read the preserved FFS data. Disable everything else up to that point and re-run the workspace.
The data is played back into the workspace and written to the output, measuring writer performance.
Assessing the time taken in transformation requires a two-step process.
First, disable writers and run the translation, taking note of the elapsed time:
Then disable the transformers, too, and rerun the workspace to calculate the time taken to read the data only:
The elapsed transformation time is the difference in elapsed time between reading the data and reading/transforming the data.
It's important not to add an Inspector or Logger transformer to see what happens to the output. This will slow the translation down and give you a false measure. You must also be sure to disable the actual writer and not just the feature types or connections to them.
The only writer helpful in this scenario is the Null format writer. This causes a writer to be present, but it does nothing except count features and discard them. The benefit is improved logging of feature counts, but without any data having to be written.
Naturally, where a FeatureReader or FeatureWriter transformer is used, assessing reading and writing requires a different procedure:
It would be necessary to run the workspace twice, once with everything disabled after that point and once with everything disabled up to (and including) that point. Then, subtract the two run times to get the correct result.
